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ABSTRACT 
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SECTION 1 


INTRODUCTION 


This report documents an investigation and evaluation of the 
use of automated tools to support programmers and analysts 
during the software development life cycle. It also pre- 
sents the results of this evaluation and makes recommenda- 
tions for future activities in this area. The work was 
performed by Computer Sciences Corporation (CSC) under the 
direction of the National Aeronautics and Space Administra- 
tion (NASA) . 

1.1 BACKGROUND AND PURPOSE OF THE EVALUATION 

Both CSC and NASA are striving for improvements in the qual- 
ity and productivity of software development efforts. In 
the past, very few automated tools have been available to 
support software requirements analysis and design. Re- 
cently, however, some tools have appeared on the market. 

The hypothesis is that the use of such tools would provide 
significant productivity and quality improvements during the 
requirements analysis and design phases of software 
development. Furthermore, improvements during these phases 
would, in turn, produce improvements in quality and 
productivity over the entire system life cycle. 

As a first step, CSC and NASA studied commercially available 
products through an industry survey. Next, a 90-day evalua- 
tion of two commercial products by programmers and analysts 
was undertaken to determine which tool is the best to sup- 
port programmers and analysts through life cycle develop- 
ment. Finally, a tool was selected for full implementation 
on the PACOR/GRO project, where complete analysis of soft- 
ware statistics over the system life cycle will determine 
whether or not quality and productivity improvements have 
actually occurred. 
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1.2 REPORT ORGANIZATION 


This report is organized into six sections and six appen- 
dixes. Section z discusses the background of the evaluation 
and includes a description of desired features as well as 
tne evaluation approach. Section 3 presents the results of 
the industry survey of currently available commercial prod- 
ucts. Section 4 documents the configuration of workstations 
installed for the in-house evaluation and CSC ' s activities 
during the evaluation. Section 5 contains the results of 
the evaluation and discusses some individual areas of inter- 
est. Section 6 summarizes the key findings from the evalua- 
tion and makes both short- and long-term recommendations for 
using automated tools. 

Appendix A shows the questionnaire used to record user re- 
sponses. Appendix B contains the requirements analysis 
problems, used in the control problem workshop. Appendixes C 
and D provide a log of hardware/software problem report sum- 
maries and evaluator suggestions. Appendixes E and F pre- 
sent recommendations from PACOR/GRO and PC&A to the Source 
Selection Board. 
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SECTION 2 - BACKGROUND OF THE PROGRAMMER/ANALYST 
WORKSTATION EVALUATION 


CSC has established a structured software development meth- 
odology, summarized in Digital System Development Methodol- 
ogy (DSDM^ ) . Part of CSC's commitment to DSDM involves the 
use of programmer/analyst workstations to allow this 
methodology to be followed easily so that programmers and 
analysts can concentrate on technical solutions to problems. 

Automated tools can replace the current mode of using paper 
models for data flow diagrams, data dictionaries, function 
specifications, structure charts, and so on. To support the 
interactive process of analysis and design, the workstation 
must be able to supply information graphically as well as in 
text form. Given the iterative nature of analysis and de- 
sign, automation and simplification of the process of gener- 
ating and refining paper models will probably increase 
efficiency. 

2.1 DESIRED FEATURES OF THE WORKSTATION 

To best support DSDM during software development, the analy- 
sis and design tools need to automate the basic steps of 
this methodology. The automated tools ultimately selected 
should oe able to 

* Implement the DeMarco structured analysis methodol- 
ogy, providing the programmer/analyst with the ca- 
pabilities to interactively 

Create and modify data flow diagrams--The sys- 
tem should be able to insert and delete 
graphic symbols for processes, data flows, 
data stores, data sources, and data sinks. 

^Version 2.0, copyright March 1984 . DSDM is a trademark 
of Computer Sciences Corporation. 
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Views of tile data flow diagram should be dis- 
played at the scale requested by the designer. 

Create an analysis data dictionary automati- 
cally derived from the data flow diagrams--For 
every modification to a data flow diagram, the 
system must 'mdate the related information in 
the data dictionary or prompt the designer for 
furtner information. 

Create and mociify process descr ipt ions--When a 
low-level process is defined or modified in a 
data flow diagram, the designer should be 
prompted for the process description. The 
system should provide a standard format for 
tne description and support the use of English 
text or decision trees. 

Create and modify data structure charts--The 
system should ensure correspondence between 
tne data dictionary and the structure cnarts 
and should prompt the designer to correct sus- 
pected inconsistencies. 

• Implement the Yourdon structured design methodolgy, 
providing the capabilities to interactively 

Create and modify the structure charts used to 
evaluate designs--All valid grapnic symools 
should be supported. A preliminary structure 
chart snould be created directly from the re- 
sults of the structured analysis process, if 
desired . 

Create and maintain a structured design data 
diet ionary--Any cnanges to either the struc- 
ture chart or the data flow diagrams should 
result in prompts for needed changes to tne 
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dictionary. The design data dictionary can be 
integrated with the analysis data dictionary. 

Describe a module's design, including a stand- 
ard format for a prolog in text and a process 
flow in program design language (PDL) --These 
module descriptions should be correlated with 
the generated structure charts. 

Construct a template for a unit test matrix 
cased on the module design 

Implement the workstation on a microcomputer--A 
basic concept for the programmer/c.nalyst worksta- 
tion is to be able to implement the tools and tech- 
niques of DSDM on a microcomputer workstation. For 
requirements analysis and design, use of a micro- 
computer is judged to be more advantageous for CSC 
than use of terminals connected to a central host 
computer. The microcomputer provides tne capabili- 
ties to 

Maintain a similar development environment 
regardless of the project's host computer-- 
This should reduce training efforts and 
learning-curve costs from one project to the 
•other 

Make the tool available on different projects 
without adding the cost of conversion 

Ensure access at all times — The project host 
computer availability is not an issue. The 
microcomputer workstation can be available 
24 hours a day, 7 days a week. Downtime or 
unavailability of a customer's mainframe 
should not be a concern during the early 
stages of development. 
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Maintain information in a standard format from 
one project to another--A project's design is 
thus maintained on a data base and can be ac- 
cessed for use on another project. The prod- 
ucts of each phase should be standard and 
should not require conversion, nor should one 
have to be concerned with the availability of 
another computer. 

2.2 APPROACH TO THE WORKSTATION EVALUATION 

CSC determined that a multiphased approach to implementing 
the workstation was the most effective plan. A multiphased 
approach allows the most thorough evaluation and helps en- 
sure a successful implementation. The three phases are 
(1) industry survey, (2) in-house evaluation, and (3) full 
implementation. Table 2-1 summarizes this plan. 

2.2.1 INDUSTRY SURVEY 

Phase 1 consisted of a two-level screening of commercially 
available products. This survey phase began with attending 
conferences, reviewing current literature on the subject, 
and consulting with technical experts to obtain a list of 
feasible sources. Initial screening consisted of telephone 
discussions and written correspondence. After the initial 
screening, products that met the following key criteria were 
considered further: requirements analysis tools, design 

tools, and microcomputer implementation. 

The second-level screening consisted of vendor demonstra- 
tions. The following features were sought and evaluated in 
this second-level screening of workstation products: 

• User friendliness 

• Graphic and textual data manipulation 

• Interactive requirements analysis tools 

• Interactive design tools 


Table 2-1. Summary of Implementation Approach 


PHASE 

CRITERIA FOR EVALUATION 

1. INDUSTRY SURVEY - 90-DAY 
DURATION 


STEP 1- INITIAL SCREENING OF 
PRODUCTS FULFILLING 
BASIC CONCEPTS 

AVAILABLE REQUIREMENTS 
ANALYSIS TOOLS 

AVAILABLE DESIGN TOOLS 


MICROCOMPUTER IMPLEMENTA- 
TION OF TOOLS 

STEP 2- PRODUCTS WITH SUFFI- 
CIENT BENEFITS FOR 
IN HOUSE EVALUATION 

75% OF DESIRED FEATURES 
SUPPORTED 

2. IN HOUSE EVALUATION - 90- DAY 
DURATION 


STEP 1- TECHNICAL PERSONNEL 
USE TOOLS TO PRODUCE 
PRODUCTS 

IMPROVED EASE FOR PRODUCING 
PRODUCTS OF DEVELOPMENT 
METHODOLOGY 


QUALITY OF PRODUCTS 

STEP 2- DIVISION TEAM 

EVALUATES BENEFITS 

QUALITY OF PRODUCTS 

COST EFFECTIVENESS OF 
PRODUCTS 

3. FULL IMPLEMENTATION - 
32-MONTH DURATION ON THE 
PACOR/GRO PROJECT 


STEP 1- EVALUATION OF STAT1S- 

' ics cc'"' ,, .et;:in cf 

EACH PHASE OF DEVEL 
OPMFNT: REQUIREMENTS 
ANALYSIS. DESIGN. 

EACH BUILD. AND 
SYSTEM TEST 

IMPROVED QUALITY OF PRODUCT 
REDUCED COST 

REDUCED TIME TO PRODUCE 
PRODUCTS 

STEP 2-EVALUATION OF 

OVER. 1 • PROJECT STA- 
TISTIC ,T COMPLETION 

OVERALL LIFE-CYCLE 
PRODUCTIVITY 

IMPROVED QUALITY OF PRODUCT 


METHODOLOGY 


CONFERENCE PARTICIPATION 
LITERATURE REVIEW 
TECHNICAL EXPERTS 


EVALUATION DURING VENDOR 
DEMONSTRATION OF PRODUCT 


USAGE OF TOOLS 
QUESTIONNAIRES 
GENERAL ASSESSMENTS 
COMPARISON OF PRODUCTS 

STUDY OF QUESTIONNAIRES AND 
TRENDS 

ASSESSMENT OF QUALITY OF 
PRODUCTS 


COMPARISON TO STATISTICS 
FROM PROJECTS IN SIMILmR 
ENVIRONMENT 

QUESTIONNAIRES TO TECHNICAL 
USERS AND MANAGERS 


COMPARISON TO STATISTICS 
AND COSTS FROM OTHER 
PROJECTS 

CUSTOMER SATISFACTION WITH 
PRODUCT 

ASSESSMENT OF DOCUMENTA- 
TION QUALITY AND COSTS 







• Usability as a development terminal on the host 
computer 

• Library capability to support software requirements 

and design tools 

• Management support tools 

• Workstation networKing capaDilities 

• Microcomputer implementation 

Tne end product of tne pnase 1 industry survey was a recom- 
mendation of two qualified products for more comprehensive 
evaluation by CSC. The criterion for oeing considered 
"qualified" was that more than three-fourths of the desired 
features be available. In addition, during the phase 1 in- 
dustry survey, initial discussions were conducted with qual- 
ified vendors regarding the possibility of CSC or vendor 
modifications to the product to meet CSC needs. 

2 . 2.2 IN -HOUSE EVALUATION 

Phase 2 consisted of a 90-day in-house evaluation of quali- 
fied products; tnis is the pnase whose activities were con- 
ducted under Task B02. During the evaluation, the tools 
were used to produce tecnnical analysis and design products 
for ongoing software development projects. 

The evaluation was conducted Dy two teams of System Sciences 
Division personnel, funded by NASA. The Division level team 
evaluated workstation usage across the Division and the 
Corporation. This team consisted of representatives of 
project management, project control, product assurance, 
software engineering, and system engineering. It was re- 
sponsiDle for assessing the overall results of using the 
tools, comparing cost effectiveness, presenting results to 
NASA and CSC managers, and negotiating with vendors. 


The second team consisted of programmers and analysts repre- 
senting six projects or areas. They were responsible for 
using the analysis and design tools of the workstation and 
evaluating the effectiveness of the tools to support DSDM. 

2.2.3 FULL IMPLEMENTATION 

Phase 3, the last step, consists of configuring the worksta- 
tions and the tools for use on a specific project. The 
project to be studied is PACOR/GRO, a 32-month software de- 
velopment project for a data capture system, work on wnich 
started in October 1984. During this period, the effective- 
ness of workstations will be assessed in a product-oriented, 
software development environment. 

Complete implementation can provide valid and reliaole met- 
rics for the entire software development life cycle. These 
metrics can then be used to determine wnetner the worksta- 
tions truly improve both productivity and quality. Results 
will be compiled at the end of each development phase (re- 
quirements analysis, design, and each build and system test) 
and at tne end of the project. The quality of tne product, 
customer satisfaction with the product, management and tech- 
nical personnel's satisfaction with tne workstation, and 
cost considerations will be evaluated in the overall assess- 
ment of the workstation. Future implementation on other 
projects will be based on the results of these evaluations. 
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SECTION 3 - RESULTS OF INDUSTRY SURVEY 


Phase 1 consisted of a two-level screening of commercially 
available pcoducts. Level 1 screening consisted of tele- 
phone discussions and written correspondence. Level 2 con- 
sisted of vendor demonstrations. 

3.1 LEVEL 1: INITIAL SCREENING 

During the initial screening, CSC found that most commer- 
cially available products support code generation and report 
writing. Products or tools that support the development 
methodology of analysis and design are fairly new. Many 
companies indicate that they are pursuing development of 
tnese tools on a microcomputer; however, relatively few 
products are available and supported today. Initially, 
eight vendors were cor.. acted whose products are currently 
available in this area. These eight products and their cur- 
rent status as analysis and design tools are listed below. 

Product Status 


Yourdon 


Tektronix 


PROMOD (GEI ) 


Excelerator 
(Index Tecn- 
nology) 

CASE 2000 
(NASTEC) 


• Not available 

• Being developed for IBM PC 

• Earliest demonstration in January 1985 

• Available for BETA test site 

• LSI or VAX based 

• U.S. availability unknown 

• IBM PC/XT or VAX based 

• Available for IBM PC/XT 


• Available on CTEC 8086 
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Product 


Status 


Boeing Argus • Package and nonsupported source avail- 

able 

• New enhanced and supported product 
availaole in January 1905 


Symbolics 


• Available on Symbolics 3600 

• No requirements analysis tools 


SOFTOOL CCC and PE • Configuration control and programming 

environment tools 

• IBM PC implementation in late 1984 

• Design environment tools in 1985 


After the initial screening, the following four products met 
tne key criteria of providing requirements analysis and de- 
sign tools and microprocessor implementation (see Sec- 
tion 2.2.1) : 

• Tektronix 

• PROMOD 

• Excelerator 

• CASE 20U0 


3.2 LEVEL 2: IDENTIFYING CANDIDATE PRODUCTS FOR FURTHER 

EVALUATION 


The second level of the industry survey was to determine 
which products that met the basic criteria provided the most 
benefits. CSC had already decided that only an in-house 
evaluation could provide a sufficiently thorough analysis of 
benefits (Phase 2). However, further information was needed 
to determine and recommend which products provided suffi- 
cient improvements over the current development approach to 
warrant tne costs associated witn an in-house evaluation. 

CSC believed that vendor demonstrations could reveal the 
availaoility of the nine desired features (Section 2.2.1) at 
this level of tne evaluation. Table 3-1 shows the desired, 
features and CSC's evaluation of the availability of each 
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Table 3-1. Desired Features as Traced in Four Products 
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feature for each product. The following subsections summa- 
rize eacn of the four products. 

3.2.1 TEKTRONIX 

The Tektronix workstation provides color graphics and joy- 
stick control and is an LSI- or VAX-based system. It re- 
ceived a 2-week evaluation in the Division facility. The 
system was developed Dy Tektronix for their in-house use and 
is now being prepared for commercial availability. The sys- 
tem will be available for Beta test-site use. 

Sever?! software problems exist with the Tektronix system. 
Because it does not yet support the generation of structure 
charts, CSC determined that the use of Tektronix at this 
point was premature. In addition, it provided less than 
75 percent of the desired features: it was not user 

friendly, did not actually support DSDM methodology, and did 
not contain management tools. 

3.Z.2 PROMOD 

PROMOD is an automated tool for requirements analysis and 
design produced by GEI in West Germany. Requirements analy- 
sis is performed using DeMarco data flow diagrams, and de- 
sign follows Modula II metnods. 

Its major disadvantage is that it imposes a strict adherence 
to tne metnodolog ies , wnich constrains the use of the tool 
for large projects due to the inherent lack of flexibility. 
In addition, the graphics interface is poor: tne size of an 

individual diagram is constrained to be the amount that can 
be drawn on a cathode ray tube (CRT) screen, and modifica- 
tion of diagrams is cumbersome. Finally, PROMOD is not cur- 
rently availaole in tne United States. 

The major advantage of PROMOD is the depth and completeness 
of its analysis function. No other product currently re- 
viewed has PROMOD' s capabilities to analyze a leveled set 
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of data flow diagrams with minispecifications and a data 
dictionary. PROMOD can produce detailed reports on the in- 
ternal consistency of data flow diagrams. 

Overall, PROMOD failed to meet CSC’s criterion of providing 
75 percent of the desired features. The design tools were 
so difficult to use tnat CSC deemed it not useful for our 
purposes. The interface was rated only partially user 
friendly. It could not currently be networked to other 
workstations to provide an automatic central project data 
Dase. CSC thus decided tnat further evaluation of this 
product was premature and not cost effective. 

3 . 2 . 3 EXCELERATOR 

Tne Index Technology Excelerator runs on an IBM PC/XT with 
special grapnic boards and a mouse interface. It produces 
good data flow diagrams and structure charts. The 
Excelerator ' s mouse interface is very user ftiendly and al- 
lows panning and zooming. The Excelerator advertises the 
following capabilities: (1) a graphic facility for data 

flow diagrams, structure charts, data model diagrams, pres- 
entation grapns, and documentation graphs; (2) a dictionary 
maintaining all system information in one place; (3) a 
screen-painting facility to develop prototypes and user in- 
terfaces; (4) a report formatter; and (5) a documentation 
facility of word processing, documentation specification, 
and production. 

The Excelerator has been commercially available since May 
1984. It provides more than bU percent of the desired fea- 
tures for the workstation. Because it runs on an IBM PC/XT, 
many management support tools are available, althougn they 
are not an integrated part of the Excelerator system. 

The main drawDack of the Excelerator is its inability to 
support the networking of workstations. Thus, no central 
pro]ect data base can be maintained automatically for 
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simultaneous users. This condition needs further consider- 
ation in CSC's evaluation. The Excelerator will provide 
this capability in a future release when IBM announces its 
networking strategy. 

3.2.4 CASE 2000 

The NASTEC CASE 2000 is a software development life cycle 
workstation implemented on the CTEC 8086. It is a powerful 
system with tools supporting all phases of development and 
the integration of tools for project management. The 
CASE 2000 advertises development tools, including structured 
design, structured programming, a requirements analysis data 
dictionary, a requirements analyzer, change control, and 
quality reviewer. It presents a life cycle manager that 
features cost/schedule estimating, task assignment and re- 
leases, and methodology/project data case and status report- 
ing. In addition, the CASE 2000 provides several 
communications protocols, compilers, editors, ana word proc- 
essors. It permits networking of up to 16 workstations. 

The primary limitation of tne CASE 2000 is tnat it is rela- 
tively difficult to learn to use. It is more powerful, how- 
ever, and tnus may be worth the learning curve. The 
CASE 2000 was not rated as user friendly, and there were 
some limitations witn drawing structure charts. It nas, 
nowever, over 80 percent of the desired features and is con- 
sidered worthwnile for further evaluation. 
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SECTION 4 


IN-HOUSE EVALUATION 


No single commercially available product met all of CSC's 
requirements, however, two products provide over 80 percent 
of the capabilities determined to be advantageous for re- 
quirements analysis and design: the Index Technology 

Excelerator and the NASTEC CASE 2000. During this evalua- 
tion, tne use of these products on a day-to-day basis al- 
lowed a valid assessment of benefits and drawbacks in the 
real world. 

4.1 WORKSTATION CONFIGURATION 

Two systems were installed in CSC's Silver Spring facility 
for the 90-day evaluation from July 30, 1984, until 
October 30, 1984. 

4.1.1 EXCELERATOR 

The Excelerator is designed to run on an IBM PC/XT. This is 
a very popular computer and widely available from a number 
of sources. It features an Intel 8088 microprocessor run- 
ning at 4.77 megahertz. Tnis is a design with a 16-bit in- 
ternal data path and an 8-bit external data path. The 
standard operating system allows the software to address up 
to 640 kilobytes of random access memory. A high-resolution 
IBM monochrome green monitor is required. Index Technology 
offers a high-resolution graphics interface as part of their 
package, which provides a bit-mapped screen with 640- by 
352-pixel resolution. An asynchronous serial interface and 
a Centronics-compatible parallel interface is provided. The 
computer provides a 5-1/4-inch floppy disk drive with a 
storage capacity of 360 kilobytes and a 10-megabyte 
Winchester hard disk. A mechanical mouse is provided for 
moving tne cursor on tne screen. Many standard parallel 
printers are supported. 
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Two Index Technology Excelerator systems were used on IBM 
PC/XTs. A mouse interface, a high-resolution graphics 
board, and Microsoft Word software were included with the 
Excelerator package. The Excelerator software can draw data 
flow diagrams and structure charts and can build a data dic- 
tionary. An Epson FX-100 printer was connected to each 
workstation. 

4.1.2 CASE 2000 

The CASE 2000 system consists of hardware developed by Con- 
vergence Technologies and software developed by NASTEC 
Corporation. A typical configuration contains an Integrated 
Work Station (IWS) master station and up to 15 IWS cluster 
stations. The master station supports a printer, external 
communications line, and a mass storage device. These are 
all accessible from any of the cluster stations. In addi- 
tion, each cluster station can support its own printer, com- 
munications line, and mass storage device. 

Each workstation (i.e., either a master station or a cluster 
station) consists of a 16-bit Intel 8086 microprocessor with 
a clock speed of 5 megahertz, a 15-inch green display screen 
that can be tilted and swiveled for user comfort, and a 
103-key detached electronic keyboard. The cluster stations 
contain 512 kilobytes of local user random access memory; 
the master station contains 768 kilobytes. The high- 
resolution character-mapped green screen displays 34 lines 
oy 132 characters; each character is contained in a 
10-by-15-pixel cell. An RS 422 channel, operating at 
307 kilobaud, tie* 3 the cluster stations to the master 
station. 

The mass storage device contains an 8-inch, single-sided, 
douole-density floppy disk drive with a storage capacity of 
one-half megabyte and a Winchester fixed disk drive with an 
unformatted capacity of 40 megabytes. Additional mass 
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storage units may be attached to the master station or tne 
cluster stations. For cluster-station-attached mass storage 
devices, only the cluster station to which they are attached 
has access to the mass storage device. 

The configuration supplied for this evaluation consisted of 
one master station, two cluster stations, a half-megaby te 
f loppy/33-megabyte hard disk mass storage system, and a 
high-quality dot matrix printer. 

Software supplied with the system consisted of the CTOS 
(Convergent Technologies Operating System) , BASE (Basic Ap- 
plication for Software Engineering) , and DesignAid software, 
the last two developed by NASTEC. CTOS is a real-time, 
message-based, event-driven, multiprogramming operating sys- 
tem. The master station provides file system and queue man- 
agement resources for all stations in the cluster. 

4.2 WORKSTATION EVALUATORS 

The evaluators consisted of two teams: one at the Division 

level and the other consisting of individuals from various 
CSC groups. 

4.2.1 DIVISION EVALUATION TEAM 

The 12-member Division evaluation team consisted of project 
managers, Division directors of product assurance and proj- 
ect control, system engineers, and software engineers. It 
met every 2 weeks to discuss activities and also met infor- 
mally with vendors as needed to discuss activities, prob- 
lems, or recommendations. The Division team reviewed input 
from users and the products of the various tools. 

4.2.2 TECHNICAL EVALUATION TEAM 

The technical team included representatives from Spacelab, 
PACOR/GRO, PC&A , the Bilateration Ranging Transponder System 
(BRTS) Scheduling Subsystem, the Network Control Center and 
the Operations Support Computing Facility (the last two as 
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participants in the control problem workshop) , and the Tech- 
nical Publications Department. Each area appointed one co- 
ordinator . 

4. 2. 2.1 Technical Coordinators 

Technical coordinators met every 2 weeks to discuss the sta- 
tus of evaluation activities. During these meetings, prob- 
lems and suggestions were discussed. Use of and access to 
workstation resources were also scheduled. 

4 . 2 . 2 . 2 Workstation Users 

Thirty-four users participated in the evaluation. These 
programmers and analysts used the workstations to perform 
their day-to-day jobs in the area of analysis and design. 

The users met every 2 weeks to provide feedback to the Divi- 
sion team and to the vendors. These meetings provided a 
forum for informal communication. 

4.3 EVALUATION METHODS 

Various methods were used to obtain objective results in the 
evaluation. The following activities were performed: 

• Questionnaires were given every 30 days to all 
users (Section 5.1). 

• A requirements analysis control problem workshop 
was conducted using the tools (Section 5.2). 

• Desired detailed features were compared (Sec- 
tion 5.3) . 

• Individual projects provided their recommendations 
(Section 5.4). 

• Tne Division evaluation team provided recommenda- 
tions based on their area of expertise (Sec- 
tion 5.5). 
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• Hardware/software problem reports were maintained 
(Appendix C) . 

• Evaluator suggestions were recorded (Appendix D) . 
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SECTION 5 - RESULTS OF IN-HOUSE EVALUATION 


During the 3-month trial period, the Index Technology 
Excelerator and NASTEC CASE 2000 workstations were made 
available to a wide variety of users. The trial period was 
divided into three segments, and the reaction of users to 
each workstation during each time segment was surveyed. 

5 . 1 QUESTIONN A IRE RESULTS AND TRENDS 

Users provided their reactions via the questionnaire shown 
in Appendix a. The questions on this form deal with user 
background, specific workstation capabilities, overall ef- 
fectiveness, and tne manner in wnicn workstations were used. 

Table 5-1 summarizes tne number of responses to the surveys 
for eacn workstation. Tne tabulations and statistics re- 
ported in this section combine data from all three surveys. 
However, only tne latest evaluation of eacn workstation from 
each participant was used in this analysis. 

Table 5-1. Survey Response 


Data Group 

Total Respondents 

Respondents Using 
Excelerator CASE 2000 

Survey 1 

19 

12 

16 

Survey 2 

23 

15 

18 

Survey 3 

13 

7 

10 

ComDined Survey 

34 

22 

29 


NOTE : Combined survey includes only the latest evaluation 

of eacn workstation from participants in any of tne 
three surveys. 

5.1.1 EVALUATOR HISTORY 

Survey organizers attempted to include the widest possible 
range of potential users. Figure 5-1 snows the distribution 
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Figure 5-1. Experience of Respondents 
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of industry and CSC experience among survey respondents. A 
broad range of experience in terms of years of experience is 
represented for both attributes. However, Table 5-2 indi- 
cates that many professional rolps are not represented in 
the sample obtained from the surveys. Survey respondents 
have been primarily programmers and analysts. Consequently, 
the surveys cannot provide much information about how well 
the workstations would support other professional roles. 

Table 5-2. Roles of Respondents 


Work Role 

Excelerator 

CASE 2000 

Division Evaluator 

U 

0 

Analyst 

11 

15 

Programmer 

8 

10 

Quality Assurance 

U 

1 

Software Manager 

2 

2 

Project Manager 

0 

0 

Support 

_1 

_1 

TOTAL 

22 

29 


Survey organizers did not assign specific problems or times 
for workstation use. Participants in the evaluation effort 
generally attempted to apply the workstations to an ongoing 
tasx. Figure 5-2 shows the distribution of contact times 
with each workstation. Relatively few users of either the 
Excelerator or the CASE 2UU0 acnieved more than 20 hours of 
contact time. Table 5-3 reports the level of familiarity 
tnat users estimated themselves to have attained. 

Only one (Excelerator ) user claimed to have become "knowl- 
edgeaole" aDout a workstation during tne 3-month evaluation 
period. Otnerwise, users of the two workstations attained 
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Figure 5-2. Exposure to Workstations 
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comparable levels of familiarity. However, a higher propor- 
tion of CASE 2000 users reported more than 20 hours of con- 
tact time. 


Table 5-3. Familiarity With Workstations 


Level of Familiarity 

Not Very Familiar 
Somewhat Familiar 
Familiar 
Knowledgeable 

TOTAL 


Excelerator 

6 

10 

5 

_1 

22 


CASE 2000 

7 
14 

8 

_0 

29 


5.1.2 TOOL CAPABILITIES 

Survey respondents rated 13 specific tool capabilities as 
well as the overall effectiveness of each workstation. 

Table 5-4 summarizes the respondent's evaluations of these 
specific tool capabilities. The Excelerator was rated sig- 
nificantly higher for ease of learning and user friendli- 
ness. No substantial differences exist between the two 
workstations with respect to ratings of requirements analy- 
sis and design capabilities. Quality assurance and project 
management capabilities were not fully explored by survey 
respondents. Users frequently complained of the lack of 
capabilities for verifying the consistency of requirements 
and design. Consequently, most survey respondents did not 
rate these capabilities. 

Survey respondents judged both workstations to be improve- 
ments over existing manual procedures, as shown in 
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Table 5-4. Evaluation of Workstatior Capabilities 


Capability 

Median 

Ratinq 3 

b 

Responses 

Excel- 

erator 

CASE 2000 

Excel- 

erator 

CASE 2000 

Graphics Support 

4 

4 

21 

27 

Easy to Learn 

4 C 

2 

22 

28 

Fast Response 

3 

4 

21 

28 

DSDM Req Analysis 

3 

3 

19 

23 

Data Flow Diagrams 

3 

3 

21 

23 

DSDM Design 

3 

3 

14 

13 

Structure Charts 

3 

4 

12 

9 

Data Dictionary 

4 

3 

16 

16 

User Friendliness 

4 C 

2 

22 

28 

Project Management 

- 

3 d 

0 

3 

Quality Assurance 

3 

3 

2 

5 

Check Requirements 

4 

3 

6 

11 

Check Design 

3 

_3 

4 

7 

TOTAL RATING 

41 

37 




a Rating: 5 = good, 1 = poor. 

Responses other than not applicable (N/A) or missing. 

Probability < 0.05 that this difference in ratings is due 
to chance. 

d 

Value not included in total rating because capability was 
not rated for both workstations. 
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Table 5-5. However, those individuals who exercised both 
systems generally stated a preference for the Excelerator. 

A significant proportion of respondents rated the 
Excelerator positively with respect to all three key attri- 
butes: quality of product, time to produce, and effort to 

generate (see Table 5-5). .The CASE 2000 received signifi- 
cant positive ratings for quality and effort only. This may 
be due to the substantially greater learning time required 
for operation of the CASE 2000 (see also Section 5.1.1). 

Table 5-5 shows that the ratings of the Excelerator tend to 
increase with increased contact, whereas those of the 
CASE 2000 tend to decrease. This suggests that some of the 
apparent attractions of the CASE 2000 weaken as the user 
becomes more familiar with the hardware. On the other hand, 
the inadequacy of the Excelerator printer was cited fre- 
quently by its users as a detriment to quality. 

5.1.3 WORKSTATION USAGE 

The two workstations appear to have been subjected to the 
same types of usage. Table 5-4 demonstrates that the same 
capabilities were exercised on both the Excelerator and 
CASE 2000. Table 5-6 shows that the same types and propor- 
tions of activities were performed on both workstations. 
Therefore, any differences in workstation ratings are not 
likely to be due to differences in workstation usage. 

Taole 5-7 shows that users of both systems generally worked 
from previously developed sketches or drafts. Somewhat dif- 
ferent results mignt have been obtained from this evaluation 
if participants had begun using the workstations at the be- 
ginning of their projects. 
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Table 5-5. Evaluation of Overall Effectiveness 


IMPROVES QUALITY? 


Workstation 

Contact 

Yes 

No 

% Yes 

Total % Yes 

Excelerator 

Low 

4 

2 

67 

83 a 


High 

6 

0 

100 

s 

CASE 2000 

Low 

6 

0 

100 

9 3 a 


High 

8 

1 

89 


REDUCES TIME? 






Workstation 

Contact 

Yes 

No 

% Yes 

Total % Yes 

Excelerator 

Low 

6 

1 

86 

92 a 


High 

5 

0 

100 


CASE 2000 

Low 

5 

1 

83 

71 


High 

5 

3 

63 


REDUCES EFFORT? 






Workstation 

Contact 

Yes 

No 

% Yes 

Total % Yes 

Excelerator 

Low 

6 

1 

86 

9 2 a 


High 

5 

0 

100 


CASE 2000 

Low 

6 

0 

‘100 

8 5 a 


High 

5 

2 

71 



Probability < 0.05 that this percentage is the result of 
random selection between YES/NO. 


Table 5-6. Type of Activity 


Workstation 

Excelerator 
CASE 2000 


Draw Diagrams 

14 

18 


Organize Data 

10 

15 
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Table 5-7. Form of Input 


Workstation 

Drafts 

Sketches 

Notes 

Mixture 

Excelerator 

6 

6 

2 

3 

CASE 2000 

8 

6 

5 

3 


The proportion of different input forms is, however, about 
the same for the two workstations. Thus, this factor prob- 
ably did not affect the relative workstation ratinqs. 

5.1.4 SUMMARY OF RESULTS 

Most of the survey respondents were programmers or analysts, 
and the workstation features most completely explored were 
those supporting the development of requirements analysis 
and design documentation (i.e., graphics, data dictionaries, 
and word processing) . Conclusions based on this survey must 
therefore be confined to how well the workstations support 
proqrammers/analysts in these activities. Two separate 
questions must be answered: How well does each workstation 

support the structured requirements analysis and desion 
methodologies? How well does that workstation support fit 
into the existing requirements/design development environ- 
ment? The survey results provide guidance on answering both 
questions. 

Overall, most survey respondents judqed the capabilities of 
the Excelerator to be superior. Although the Excelerator 
was rated significantly higher in terms of ease of learninq 
and user friendliness, the two systems were not rated very 
differently in terms of support for requirements analysis 
and desiqn. Both systems appeared to offer improvements 
with respect to the key attributes of quality, time, and 
effort. However, a high percentage of respondents rated the 
Excelerator as likely to be more beneficial. 
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The ease of inteqrating either workstation into an existinq 
requirements/desiqn environment depends on its match to that 
environment. The Excelerator and CASE 2000 are optimized 
for different environme rtz. The former tarqets the environ- 
ment in which many unrelated, small- to medium-scale 
requirements/design problems are being solved simultane- 
ously. The latter tarqets the environment in which the so- 
lution to a sinqle larqe requirements/desiqn problem is 
developed over a relatively Iona period of time. 

The Excelerator ' s ease of learninq and operation (via a 
mouse) makes the system cost effective in those situations 
in which one or two individuals spend a few months producing 
a formal requirements/desiqn specification (possibly based 
on input from a larger team) . These individuals spend the 
rest of their time on other activities (e.q., mathematical 
analysis or proqramminq) . The provision for individual 
diskettes allows the system to be shared by many users with 
different problems. Furthermore, the computer can be used 
to run other software when no requirements/desiqn activity 
is in progress. 

The CASE 2000' s central disk and data dictionary support the 
situation in which many individuals are workinq on different 
aspects of the same requirements/desiqn problem. This sys- 
tem simplifies configuration management for larqe projects. 
The additional cost imposed by the lenqthy traininq and 
phase-in period are recovered durinq the relatively lonq 
development period; function keys move the user through the 
system faster than does a mouse. Furthermore, the function 
keys can be programmed to satisfy project-specific needs. 
However, "difficult to learn" implies "easy to forqet," so 
this system is not suited to non-full-time users. The over- 
all hiqher ratinq, lower cost, and potential for other uses 
suqqests tnat the Excelerator is preferable to the CASE 2000 
except for larqe projects. 


5.2 CONTROL PROBLEM WORKSHOP 


A 2-week structured analysis workshop was conducted during 
the evaluation period. Section 5.2.1 describes the workshop 
activities, and results are presented in Section 5.2.2. 

5.2.1 WORKSHOP ACTIVITIES 

Eight software analysts were selected to use the worksta- 
tions for defining the software requirements for a control 
problem. Two members of CSC's Division evaluation team se- 
lected the problem, directed the efforts of tne analysts, 
evaluated the results, and collected the analysts' personal 
evaluations of the control problem exercise and the work- 
stations. 

The analysts were selected by the CSC managers of two major 
NASa/CSC pro jects--the Network Control Center and the Opera- 
tions Support Computing Facility. All the analysts were 
currently performing analysis efforts on their projects and 
would be able to use what they learned from the exercise in 
the near term. One of the analysts had prior experience 
using both workstations. None of the others had any such 
experience. All analysts had prior experience using the 
structured analysis method (DeMarco, Gaines, Sarson) sup- 
ported by both workstations. However, none of the analysts 
were experts in the method. 

The Division evaluation team elected to establish three con- 
trol groups for the exercise. One group (two analysts) 
would use the Excelerator workstation; another group (three 
analysts) would use the CASE 2000 workstation; and a third 
group (three analysts) would use manual methods to solve the 
problem, but would have access to a sophisticated word proc- 
essor to build and maintain text. The word processor given 
to the third group was that provided on the CASE 2000 work- 
station. This group was not given access to the CASE 2000 
graphics support capabilities. 
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The control problem exercise spanned 2 weeks (10 business 
days) . The analysts were assigned to the exercise on a 
half-time basis, i.e., from 1:00 p.m. to 5:00 p.m. on each 
of the 10 business days. The day- to day scenario was as 
follows : 

Day 1 . The Division evaluation team conducted a 4-hour 
seminar on the analysis method to be used during the exer- 
cise. The analysts were assigned to the three groups. The 
groups were structured so that the average experience of 
each qroup was even. Each group was told to function as an 
analysis team, i.e., only one integrated set of analysis 
products was to be produced by each qroup. The analysts 
were given a training problem to be used durinq the next 
4 days while learninq how to use the workstations. (The 
traininq problem is presented in Appendix B.) 

Day 2 . The Excelerator and CASE 2000 groups were qiven 
hands-on instruction on using the workstations. The in- 
structions were given by vendor representatives and experi- 
enced CSC users of the workstations. The Excelerator 
instruction period lasted about 2 hours, whereas the 
CASE 2000 instruction period lasted all afternoon 
(4 hours) . The Excelerator qroup beqan using their worksta- 
tions to solve the training problem. 

Day 3 . The word processor qroup was qiven about 2 hours 
of instruction on usinq the word processor assiqned to 
them. The CASE 2000 qroup used their workstations for the 
remaining 2 hours. The Excelerator qroup used their work- 
stations for 4 hours. 

Day 4 . All three groups were qiven 4 hours' access to 
the workstations for the training problem. The CASE 2000 
and word processor qroups had to share their workstations 
since they were using the same equipment. 
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Day 5 . All three qroups were given 3 hours' access to 
the workstations for the training problem. During the last 
hour of the day, the analysts and the Division evaluation 
team met to discuss problems and to review the control prob- 
lem to be used during the next week. (The control problem 
is presented in Appendix B.) Two experienced users of the 
workstations also attended the meeting to help solve prob- 
lems reported by the analysts. The Division evaluation team 
gave each analyst a copy of the control problem and ex- 
plained the intent of the problem. 

Days 6-7 . The workstations were assigned to the three 
analyst groups for 4 hours each day. The Division evalua- 
tion team talked to the analysts while they were usinq the 
workstations to identify the problems the analysts were ex- 
periencing in usinq the workstations or understanding the 
control problem. Other experienced users of the worksta- 
tions helped the analysts solve problems they were havinq 
with the workstations. The Division evaluation team an- 
swered all questions relevant to the control problem it- 
self. At the end of day 7, each group provided a "first 
cut" of their control problem solutions to the Division 
evaluation team for inspection. 

Day 8 . Durinq the morninq, the Division evaluation team 
inspected the control problem solutions provided by the 
three analyst qroups. The activities of the control group 
on this day were similar to those of the preceding two days, 
with one exception. The Division evaluation team met with 
each group separately to inform the analysts of problems in 
their solutions that had to be corrected. In addition, one 
of the requirements in the original problem was inten- 
tionally chanqed. The intent of these two actions by the 
evaluation team was to simulate two realistic situations: 
chanqinq analysis products to brinq them in line with stand- 
ards and responding to requirements chanqes. This also 
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forced the analysts to use the workstations to chanqe the 
products built on the workstations. 

Day 9 . The activities of this day were basically the 
same as those on days 6 and 7. 

Day 10 . The activities of this day were similar to 
those of the preceding 4 days, except that only 3 hours were 
spent usinq the workstations. Durinq the last hour of the 
afternoon, the analysts and the Division evaluation team met 
to review the results of the entire exercise. Each qroup 
provided a copy of their analysts' products to the Division 
evaluation team. The analysts were asked for an initial 
assessment of the exercise in terms of the value of the 
workstations and also the value of the traininq received in 
the analysis method. All analysts aqreed that automated 
support noticeably improved productivity and quality. In 
addition, all aqreed that their understanding and apprecia- 
tion of the analysis method used was siqnif icantly in- 
creased. The analysts were then directed to prepare an 
informal report detailinq their assessment of the worksta- 
tions. They were qiven one week to prepare this report. 

5.2.2 CONTROL PROBLEM RESULTS 

All three teams completed the control problem on time. Each 
team produced a credible set of uocumentat ion that was of 
sufficient quality to be used in an informal review. The 
team personnel felt that use of the workstations provided a 
noticeable increase in productivity, but there is no quanti- 
tative data to support this observation. The team personnel 
also felt that the use of any automation aid (e.q., the word 
processing software of the word processing team) was helpful 
in increasing the ease of modification and technical quality 
of the products. 
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The overall ranking of the workstations from control problem 
data was as follows: 


Rank 


System 


1 

2 

3 


CASE 2000 
Excelerator 

Word processing software 


Tne difference between the top and bottom in terms of work 
product results and user satisfaction was not great and 
could not be the basis for a recommendation of one product 
over the otner. Additionally, an inability to show a clear 
superiority of work products between the full workstation 
use and just the word processor points up tne need to con- 
tinue to analyze these workstations under more realistic 
work conditions. 


The following sections present more detailed results for 
each control proDlem team. 

5. 2. 2.1 CASE 200U 

Tne CASE 2U00 team generally felt that the workstation with 
the DesignAid product was very helpful in performing the 
requirements analysis problem. The team suggested the fol- 
lowing improvements to make tne CASE 2000 even more useful: 

• Capability to draw curved lines on data flow dia- 
grams 

• Capaoility to include text from supplemented files 
tnat descrioes a data dictionary entry in tne data 
dictionary report 

• Capaoility to have invisible sources and sinks for 
data flows 

• A better printer for final copies of printouts 


5-15 


9009 


• Capability to use multiple lines for data flow 
names on data flow diagrams 

• Capability to "auto-def ine" items in the data dic- 
tionary from a data flow diagram 

• More generalized report generation capabilities 

The most undesirable features of the CASE 2000 were the dif- 
ficulty in learning to use the system and the poor user in- 
terface, especially in the complexity of key sequences 
necessary to accomplish certain functions. 

5. 2. 2. 2 Excelerator 

The Excelerator team was less impressed with their worksta- 
tion, although they felt that, with suitable enhancements, 
the workstation would definitely help an analyst. Major 
desired improvements were as follows: 

• More control over the data dictionary report writer 

• Addition of cross-references between the data dic- 
tionary and data flow diagrams 

• Capability to print the additional text portion of 
a description form 

• Capability to move labels on data flow diagrams 
without retyping them 

• Less sensitive mouse for performing certain opera- 
tions 

• Capability to have invisible sources and sinks for 
data flows 

• Having the display match the printed output for 
data flow diagrams (what you see is what you qet) 

This team felt that the user interface for th*' Excelerator 
was good, although one member experienced problems with the 
mouse when trying to manipulate user labels. 

5-16 


9809 


5. 2. 2. 3 Manual 


This group drew data flow diagrams by hand and used the 
CASE 2000 word processor to generate the data dictionary and 
function specitications. For a problem of this size, their 
products were as good as those generated using the full ca- 
pabilities of the workstation. For a much larger problem 
(one more typical of what CSC might do on a contract) , this 
would likely not be the case. All members of the group felt 
that the use of the word processor improved the quality of 
their products and their ability to maintain those prod- 
ucts. Their recommendations for improvement were in the 
area of increasing automation support. Specifically, they 
desired 

• A togqle switch for insert-ver sus-overstr ike mode 
in the word processor 

• A sorting capability that is insensitive to alpha- 
betic case 

• The capability to sort entire paragraphs instead of 
merely lines (to keep extended definitions together) 

5. 3 COMPARISON OF FEATURES 

A detailed comparison of features available on the 
Excelerator and the CASE 2000 to support requirements analy- 
sis and desiqn was undertaken as a part of this evaluation. 
Section 5.3.1 outlines the approach used; Section 5.3.2 pre- 
sents the results of the comparison; and Section 5.3.3 sum- 
marizes the conclusions. 

5.3.1 APPROACH 

Eiqht major categories of features were identified; 

• Data flow diagrams 

• Structure charts 

• Data dictionary 
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• Function specifications 

• Data flow diagram validation 

• Structure chart validation 

• Report/display generation 

• General/other 

The eight categories were assigned relative weights adding 
up to 100. Each major category was further divided into 
specific features. Each feature was assigned a weight of 
either 1 (desirable) or 2 (mandatory) . Four groups who had 
used both the Excelerator and the CASE 2000 fairly exten- 
sively during the evaluation period were asked to assess the 
two systems feature by feature. This input was used to as- 
sign numerical scores for each feature on a scale of 0 to 5 
(0 = not available, 1 = low, 5 = high). 

A final score for each workstation was achieved as follows: 

Let w^ = weight of ith feature in a major category (value = 

1 or 2) 

r^ = raw score for ith feature (range = 0 to 5) 

Wj = weight of jth major category (LWj = 100) 

Step 1 . For each feature, compute weighted score (s i ) as 
follows: 


Step 2 . For each major category, compute overall raw score 
(Rj) as follows: 


R 


j 
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Step 3 . For each major featuL*?, compute overall weiqhted 


score (Sj) 


as follows: 



Step 4 . Compute final scores for each workstation as 
follows: 


“ToT5 


The range of final sc r^s is 0 to 5. 

5.3.2 RESULTS 

Table 5-8 lists scores for individual features and shows the 
computation of overall raw scores for each major cateqory. 
Table 5-9 shows the computation of final scores from overall 
raw scores. In summary, the Excelerator and the CASE 2000 
scored as follows: 

Excelerator . 2.01/5.00 

CASE 2000 2.82/5.00 

5.3.3 CONCLUSION 

Two conclusions emerqe from the precedinq analysis: 

• Neither the Excelerator nor the CASE 2000 scored 
very hiqh. This indicates that both systems lack 
many of the desired features. 

• Feature for feature, the CASE 2000 is superior to 
the Excelerator. 
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Table 5 8. Computation of Overall Raw 
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Table 5-8. Computation of Overall Raw 
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Table 5-8. Computation of Overall Raw Scores (8 of 8) 
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Table 5-9. Computation of Final Scores 



















5.4 WORKSTATION USAGE DURING THE EVALUATION PHASE 


5.4.1 SPACELAB 

Two Spacelab project teams were used to develop data flow 
diaqrams and their associated data dictionaries and func- 
tional specifications for the Spacelab Input Processinq Sys- 
tem (SIPS) . Each team consisted of four proqrammer/analysts 
with varyinq levels of skill and a balanced skill mix. 

The work was performed over a 2-month period (Auqust throuqh 
September 1984) , with participants spendinq an averaqe of 
approximately 25 percent of their time on this activity. 

This includes the time required for traininq and familiari- 
zation. 

Weekly technical review meetinqs were held to discuss and 
critique requirements analysis products developed to date. 
Thus, the final products evolved throuqh numerous iterative 
analysis cycles, much as they would in an actual software 
development environment. The meetinqs also coordinated ef- 
forts between the two teams, and work assiqnments to team 
members were made at that time. Input to the workstation 
evaluation process was provided by havinq each participant 
periodically complete survey questionnaires and attend bi- 
weekly user qroup meetinqs. 

All participants found the workstations to be an improvement 
over the manual approach and tLe quality of products from 
both workstations to be qood. The CASE 2000 team required a 
lonqer time for traininq and learninq before participants 
necame fully productive. The Excelerator was felt to oe 
siqnif icantly easier to use. However, even for a small ef- 
fort such as this, the availability of a centralized data 
base on the CASE 2000 was found to be useful. The manual 
merqinq of files across workstations was considered tedious 
and errorprone on the Excelerator. Both workstation systems 
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were felt to be weak in data dictionary and validation 
support. 

5.4.2 PACOR/GRO 

The PACOR/GRO project is the target project for the first 
in-depth, product-oriented, long-term use of the programmer/ 
analyst workstation. During the first 2 months of the eval- 
uation period, the PACOR/GRO project began as a small task 
to develop software requirements for the PACOR/GRO software 
system. This task required a preliminary delivery of the 
Software Requirements Specification on a Programmer/Analyst 
Workstation on September 28, 1984. The PACOR/GRO project 
officially began on October 1, 1984. 

The PACOR/GRO development team's use of the workstation re- 
flected the small size and nondedicated nature of its origi- 
nal staff and tne immediacy of its deliverable. Their 
experience with the workstation can be divided into four 
sequential phases: 

• Familiarization with the CASE 2000 

• Fami iarization with the Excelerator 

• Product development and delivery on the Excelerator 

• Product development on the CASE 2000 

The familiarization with the CASE 2000 took place over a 
3-week period. This process consisted of attending vendor 
classes, taking the tutorial, ^nd independent experimenta- 
tion. Three members of the PACOR/GRO task participated. 

The familiarization with the Excelerator took place over a 
1-week period and consisted of the tutorial and independent 
experimentation. Three members of the PACOR/GRO task par- 
ticipated . 

At this point, the preliminary PACOR/GRO Software Require- 
ments Specification was developed on the Excelerator sys- 
tem. This effort consisted of the development of data flow 
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diagrams, process descriptions, a data dictionary, and 
text. Intermediate products were reviewed with the custo- 
mer. At the end of the task, a 75-paqe document produced 
directly from the output of the Excelerator was delivered to 
the customer. The quality of this delivery was niqh, and 
the experience with the Excelerator product positive. Four 
members of the PACOR/GRO development team participated in 
this full-time effort over a 5-week period. 

Immediately following the initial delivery, and coincident 
with the formal start of the PACOR/GRO project, the material 
was transported to the CASE 2000 from the Excelerator with 
assistance from NASTEC personnel. A two-workstation config- 
uration was made available exclusively to the PACOR/GRO 
team. Onsite technical assistance was provided for the 
first week. The PACOR/GRO team, now augmented to six, con- 
tinued the structural analysis activity on the CASE 2000. 
Intermediate products were reviewed with the customer. The 
experience with the CASE 2000 was extremely positive. 

As more staff members were added to the project, the per- 
ceived advantages of the CASE 2000 centralized data base 
increased. The data dictionary for the rudimentary Software 
Requirements Specification had already exceeded the floppy 
disk on the Excelerator; this limitation did not exist on 
the CASE 2000. The ability of the CASE 2000 to mix text and 
qraphics was a tremendous advantage in document production. 
Overall, it appears to be a more flexible, sophisticated 
tool than the Excelerator. 

The PACOR/GRO project's experience with both workstations is 
positive. The CASE 2000 is, however, more suited to PACOR/ 
GRO project needs. 
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5.4.3 PC&A 


PC&A personnel used both workstation systems to support five 
different software development activities ranqinq from con- 
cept evaluation to preliminary desiqn and prototypinq. Al- 
thouqh the evaluation period was inconveniently timed for 
some efforts, several deliverable items were produced (in 
part) with the workstations, and considerable information 
was derived from their use. 

Task members of the qraphics assessment task were trained on 
both systems to investiqate interactive qraphics techniques 
and potential. This task is responsible for advisinq on 
needed qraphics capabilities, in the fliqht dynamics environ- 
ment. Task personnel were primarily interested in editinq 
and displayinq and in interaction methods on the worksta- 
tions; they did not investiqate the mechanized formalisms 
provided by these systems. 

The CASE 2000 system was used by the Software Enqineerinq 
Laboratory (SEL) data base task personnel to perform and 
document a data flow analysis of an existing data 
collection/validation process. Althouqh limitations of the 
data dictionary and constraints of conforming to certain 
rules precluded the use of the desiqn aid software, the data 
flow diaqram capabilities of the system were exercised dur- 
inq a moderately riqorous create/review/correct editinq 
process; the data flow diaqrams were used in an informa- 
tional memorandum delivered to GSFC. 

Both workstation systems were used by Desiqn Metrics task 
personnel in recastinq the requirements specification for an 
Earth Radiation Budqet Satellite (ERBS) control utility into 
a more riqorous form. This effort involved a restructur inq 
of text-and-f ormula specifications in a composite model of 
data flows, entity-relationship descriptions, and state 
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models to capture all aspects of the utility. Task person- 
nel were required to make extensive use of the analytical 
and qraphic capabilities of the workstations. 

The Attitude Determination Error Analysis System (ADEAS) 
task is presently in the preliminary desiqn phase for this 
support software system. Task personnel were trained on 
both workstations and have made some investigations int-o 
capabilities. More extensive use would have been made, but 
the evaluation period conflicted with ADEAS desiqn phase 
activities. 

Task personnel on the Flight Dynamics Analysis System (FDAS) 
task (with some overlap with the ADEAS and Desiqn Metrics 
activities) attempted to use the workstation systems to pro- 
totype screen-sequencinq user interfaces. The FDAS require- 
ments specification effort is using an operational 
specifications approach to which such a prototype would be 
well suited. Neither workstation system, unfortunately, 
provided adequate capabilities for a full-scale prototype 
effort. The attempt was abandoned. 

In addition to these specific projects, a number of other 
PC&A personnel made minor use of the workstation systems out 
of curiosity and discussed their findings with persons more 
directly involved. 

5.4.4 BRTS SCHEDULING SUBSYSTEM (BSS) 

The BRTS Scheduling Subsystem (BSS) was well into the pre- 
liminary design phase of the software development cycle. 

The Preliminary Desiqn Review ( PDR) was postponed 3 weeks to 
allow task members to produce some of the desiqn products on 
the workstations and at the same time to assist in evaluat- 
ing the workstations. Oriqinally, the plan was to produce 
the same output on both machines, but insufficient time pro- 
hibited this. Because the CASE 2000 was flexible enouqh to 
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allow users to define their own symbols, it was selected as 
the tool best suited to BSS needs. 

An attempt was made to produce data flow diagrams and the 
accompanying data dictionary. The limited data flow concept 
on the CASE 2000 did not support current standards incorpo- 
rated into our design. Current solutions were not satisfac- 
tory and would have increased the clutter of the data flow 
diagrams. Entering the data dictionary would have been te- 
dious and would not have improved upon an in-house product 
already in use. It was then decided to produce structure 
charts, interface tables, and the accompanying data diction- 
ary at the workstation. The process was tedious initially, 
but improved with experience. Use of function keys and cre- 
ation of symbol files also helped. 

Task members did not spend much time on the Excelerator but 
did enough to determine that its main advantage over the 
CASE 2000 was its user friendliness. 

5.4.5 TECHNICAL PUBLICATIONS 

The Technical Publications Department's graphics supervisor 
was chosen to use and evaluate the workstations. The evalu- 
ation consisted of preparing some routine types of artwork 
that are presently prepared in the traditional pen-and-ink/ 
type-pasteup method, which involves two distinct functions 
as well as two individuals with vastly different skills. 

Four types of art were created: an organization chart, data 
flow diagrams (two) , a structure chart, and a milestone 
chart. An evaluation of the workstations' capabilities 
follows . 

• CASE 2Q00 --In general, the CASE 2000 was found to 
oe fast and easy to use. Graphics were fairly easy to enter 
and modify. However, standard-size symbols were too small 
for type size. Correcting curved portions of symbols was 
tedious and often required replacement of entire symbols. 
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Simple milestone charts were easily drawn but space for type 
could be a problem on more complex charts. In addition, the 
maximum timespan on a milestone chart is restricted to 
2ti months unless months are stacked or changed to one- 
character designations. As a self-training tool, the 
CASE 2000 tutorial is inadequate and confusing, and, accord- 
ing to a NASTEC representative, inaccurate in places. 

• Excelerator --The rnenu-driven Excelerator was in 

some .respects easier to use than the CASE 2000 because the 
user continually receives prompts. Furthermore, the use of 
a mouse makes cursor movement faster. However, on the 
Excelerator, no means exist for drawing a line unless it is 
a connection between two symbols. Also, type is very con- 
densed and difficult to read and not centered vertically 
within tne symbol. Overall, the major drawback of the 
Excelerator is that it is far less flexiDle in freehand 
drawing than is the CASE 2000. 

5.5 DIVISION EVALUATION 

This section presents the Division evaluation team's assess- 
ment of avaiiaole tools. Section 5.5.1 details tne tool 
evaluation; Section 5.5.2 addresses performance and configu- 
ration considerations, including an estimation of PACOR/GRO 
sizing requirements; and Section 5.5.3 presents cost data 
for tne CASE 2000 and the Excelerator. 

5.5.1 EVALUATION OF TOOLS 

5. 5.1.1 Requirements Analysis 

The Excelerator, the CASE 2000, and sophisticated word proc- 
essors are ail valuable aids for requirements analysis and 
represent major improvements over purely manual methods. 

For projects on wnicn tne analysis team is small (one to 
three persons) , all three tools will have equal value. For 
projects with a medium-sized analysis team (tnree to five 
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persons) , the Excelerator and CASE 2000 workstations would 
probably be much better than a word processor alone. For 
projects with larqe analysis teams (more than five anal- 
ysts) , the CASE 2000 would probably be superior to the 
Excelerator, primarily because of the availability of the 
controllable, shared data base on the CASE 2000. 

When the evaluation effort beqan, we expected to see im- 
proved technical quality in the analysis products. We were 
not sure if productivity would also improve. The results, 
however, seemed to indicate a noticeable improvement in both 
quality and productivity. 

The most important capabilities offered by the Excelerator 
and CASE 2000 workstations relate to the preparation, main- 
tenance, validation, and control of data flow diaqrams and 
data dictionaries. Both workstations offer similar capa- 
bilities. Both can be used to prepare data flow diaqrams 
and data dictionary entries. Preparinq data flow diaqrams 
was easier to learn and use on the Excelerator than on the 
CASE 2000 workstation. The Excelerator is more flexible in 
terms of the structure and lenqth of data dictionary entries 
than the CASE 2000. 

Both workstations provide adequate capabilities in maintain- 
inq data flow diagrams and data dictionaries. The ease of 
maintenance seemed to be the major factor that improved the 
quality of the analysis products. With automated support, 
users are far more willinq to chanqe the products to improve 
technical quality than if they had to rely only on manual 
methods. With automated tools, users seem to view their 
analysis products with less ego and thus are less likely to 
resist makina chanqes to the products. 

Both workstations offer limited validation capabilities for 
data flow diaqrams and data dictionary entries. The 
CASE 2000 validation capabilities are better than those 
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offered by the Excelerator. Validation capabilities are 
just as important as those for preparing and maintaining 
data flow diagrams and data dictionaries. This is the area 
in which improvement is most needed on both workstations. 

The Excelerator has very limited capabilities for control- 
ling the content of analysis products. The CASE 2000 has a 
much better control capability; however, it should be fur- 
ther improved. Although each project using the CASE 2000 
can have its own controlled data base of data flow diagrams 
and data dictionary, only one data base can be active at any 
one time at all active terminals. What is needed is a ca- 
pability to designate, by active terminal, which data base 
will be accessed from the terminal. This would allow multi- 
ple projects to use a cluster of terminals concurrently. 

This capability would also allow individual analysts to pre- 
pare interim data flow diagrams and data dictionary entries 
more quickly, without having to constantly worry about nam- 
ing conflicts with other analysts using the workstation. 
Thus, the analysts could negotiate name chanqes among them- 
selves after they have completed their interim analysis 
products. 

Either workstation, or a word processor, is far better than 
purely manual methods. Improved quality and productivity in 
requirements analysis will occur if the analysts on a proj- 
ect have ready access to such tools. 

5. 5 . 1. 2 Design 

The Excelerator and CASE 2000 workstations are valuable aids 
for system design efforts and represent major improvements 
over purely manual methods. Either workstation would be 
suitable for small to medium system desiqn teams (up to six 
designers) . For large system design teams (more than six 
designers) , the CASE 2000 workstation would probably be 


5-37 


9309 


superior, primarily because of the availability of a con- 
trollaole, shared data base on this workstation. 

Although we did not study the value of a word processor in 
design efforts, it is felt that productivity and quality 
would also improve if the system design team were to have 
ready access to word processors. However, tne increased 
quality and productivity probably would not be as great as 
witn tne Excelerator or CASE 20U0 workstations. Word proc- 
essors would be very useful in managing a design data dic- 
tionary. On one recent project, a word processor proved 
very useful in detecting duplicate and incorrectly named 
units on structure cnarts. 

The most important capabilities offered by the Excelerator 
and CASE 20U0 workstations relative to design efforts deal 
with the preparation, maintenance, and control of structure 
charts and design data dictionaries. Both workstations pro- 
vide adequate capabilities in preparing and maintaining 
structure charts and data dictionaries. The Excelerator is 
easier to learn and use than the CASE 2000. The CASE 2000 
workstation offers an automatic means of controlling struc- 
ture cnart and data dictionary content, whereas the 
Excelerator 1 s control capabilities depended on manual proce- 
dures. However, the CASE 2000 has the same control defi- 
ciencies, as descrioed in Section 5. 5. 1.1. 

Neither workstation provided adequate structure chart vali- 
dation capaDilit ies. At oest, the workstations could only 
detect when a unit interface item (a data or control couple) 
was not defined in the data dictionary. The validation ca- 
pabilities need to be strengthened considerably, especially 
those related to detecting inconsistencies in interfaces 
oetween calling and called units (modules) . 

Both workstations treated the products of analysis and de- 
sign as two independent sets of products. The value of the 
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workstations would be considerably increased if they could 
transform data flow diagrams into structure cnarts. Neither 
workstation offers such a capability, in even a limited 
fashion. If such a capability existed, a designer could use 
the computer-generated structure charts as a check on the 
designer's structure charts. Such a capability would not be 
easy to provide and would probably require the designer to 
provide some guidance to the workstation software. This 
capability may, however, be worth the potential development 
cost. 

5. 5. 1.3 Development Life Cycle 

Other than analysis and design capabilities, the Excelerator 
provides no features to ease the preparation of other life 
cycle products. The Excelerator is, however, based on a 
quality word processor, Microsoft Word. This word processor 
is easy to use and is one of the most sophisticated 
available on any computer. It could tnus be useful in 
generating most of the technical documentation prepared 
during the development life cycle, e.g., user guides, test 
plans and procedures, and technical reports. 

Because the Excelerator runs on an IBM PC, the PC itself 
could be used to generate unit prologs, PDL, and code. If 
the PC were linked to a mainframe computer, the code could 
be shipped to the host computer for testing. Compilers ex- 
ist on the PC for most coding languages used in NASA soft- 
ware products. Compilation errors could thus be eliminated 
before the code is shipped to the host computer. Some 
limited unit testing could also be done on the PC. These 
extended capabilities would, however, require the purchase 
of additional hardware and software. 

Because the CASE 2000 workstation is based on Convergent 
Technology workstations, it is designed to provide the host 
computer interface specified above. Language compilers for 
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most NASA codinq lanquaqes can also be purchased for the 
workstation. The CASE 2000 also offers a sophisticated word 
processor with many capabilities similar to those provided 
by Microsoft Word on the PC. Thus, both workstations could 
be extended to provide useful tools needed durinq software 
implementation. 

With no additional investment, both workstations provide a 
good word processing capability for documentation produced 
throughout the development life cycle. With some additional 
investment, both could also provide support for software 
implementation efforts. 

5. 5.1.4 Quality Assurance 

The quality of most products produced while usinq the work- 
stations was noticeably hiqher than was experienced on other 
projects usinq purely manual methods. This quality improve- 
ment applied both to products that had not yet been reviewed 
by quality assurance personnel and to those chanqed to cor- 
rect quality problems detected by quality assurance person- 
nel. The probable reason for these improvements was the 
ease with which users could make changes in their products. 
They were more likely to take the time to make voluntary 
chanqes that improved quality and were less likely to debate 
the merits of quality problems detected by quality assurance 
personnel. The users simply appeared to have less eqo in- 
vested in their products. 

In the past, quality assurance personnel checked development 
products in hardcopy form. Ideally, in an automated envi- 
ronment offered by workstations, quality assurance personnel 
would use the workstations to check product quality and thus 
would not need hardcopy versions of the products. It is not 
yet clear, however, how this would actually occur. Hardcopy 
versions of the products existinq on the workstations must 
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still be used for quality inspections. If, however, work- 
stations provided an easy way to annotate the products, the 
reliance on hardcopy output could be reduced significantly. 

The workstations should provide an easy way to access the 
information data base created by the workstation software. 
Quality problems that often go unnoticed become obvious when 
information is sorted or looked at in a different way. The 
workstations should therefore provide tools that could be 
used as building blocks for viewing, sorting, and otherwise 
restructuring a workstation's information data base. 

5. 5. 1.5 Project Management 

Based on presentations by Index Technology, the Excelerator 
currently has no implemented project management capabili- 
ties. However, a review of early requirements statements 
indicates that the company understands the need for an inte- 
grated project management capability. As Index Technology 
further develops their requirements, a careful review should 
be made to bias the implementation toward management of sys- 
tems development projects. 

Although numerous project manag^^ent packages are available 
for the IBM PC, no package has yet interfaced with the 
Excelerator. Ideally, project management and project per- 
formance tools should be integrated through a common data 
base. It is unlikely that a vendor other than Index Tech- 
nology could produce such an integrated set of tools. 

A version of the Life Cycle Manager was reviewed during 
presentations made by NASTEC in August. The Life Cycle Man- 
ager appears to be designed toward assisting the planning of 
small efforts, and as such has limited flexibility. Al- 
though some of the concepts are sound (especially a common 
data base of completed events) , it does not permit schedul- 
ing of dependencies nor lend itself to performance measure- 
ment. The current reports are inadequate both in content 
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and organization. Extensive work would be necessary to use 
the Life Cycle Manaqer as a tool for manaqinq systems devel- 
opment projects. 

5.5.2 PERFORMANCE AND CONFIGURATION CONSIDERATIONS 

Sections 5. 5. 2.1 and 5.5. 2. 2 outline the performance and 
configuration considerations for the Excelerator and the 
CASE 2000, respectively. 

5. 5. 2.1 Excelerator 

As delivered, Index Technology assumes that you have an 
IBM PC/XT with an IBM keyboard, the IBM monochrome monitor, 
and an asynchronous communications serial interface. To 
this, Index Technology adds the Tecmar multifunction board 
with 128 kilobytes of additional random access memory, a 
serial and parallel I/O port, the Tecmar high-resolution 
qraphics board, a Microsoft mouse, the Excelerator software, 
and Microsoft Word word processing software. A dot matrix 
printer or plotter must also be added. The product supports 
such printers as the Epson FX-100, the IDS Prism printer, 
the IBM qraphics printer, and the Toshiba P1351 printer. In 
a comparative test, all of these printers printed a standard 
flow chart in from 7-1/2 to 8-1/2 minutes. The Excelerator 
also supports plotters such as the HP 7J75A, which took 
8 minutes to print the standard test diaqram. For signifi- 
cantly faster printout, a QMS Laser Graphics printer may be 
used. This printed the standard diaqram in 1 minute. From 
a performance perspective, printout speed is very signifi- 
cant, because the product is desiqned such that print spool- 
ing is not supported. Thus, the system cannot be used for 
anything else until a requested printout is finished print- 
ing. This is judged to be a severe limitation on the per- 
formance of the system. 

Because the Excelerator, in its current form, is a sinqle 
station system, it presents significant problems for use on 
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large projects. Other than copying files to a floppy disk 
and loading them onto a second system, tnere is no way for 
two users to work from the same data base at the same time. 
No easy merging of files is supported. At some later time, 
Index Technology plans to support multiple systems inter* 
faced through a local area network, however, this is not 
presently available. 

For improved operacing performance, the Excelerator can now 
be installed on an IBM PC/AT. This system uses an 
Intel 80286 microprocessor, which is a true 16-bit CPU with 
an internal and external 16-bit data bus. For internal cal- 
culations, a factor of three improvement over an IBM PC/XT 
nds oeen reported. The printout speed limitations mentioned 
previously still exist with this configuration. 

5.5.2. 2 CASE 2UUU 

The CASE 200U system (Section 4.1.1) consists of a series f 
clustered worKstations, eacn with its own 16-bit microproc- 
essor running at 5 meganertz. Local memory of naif a mega- 
oyte is used at eacn station for local program processing. 
Eacn station has a copy of tne operating system for local 
execution. Very nigh performance at each station results 
front this architecture because eacn user has his/her own 
powerful CPU and one-half megaoyte of local memory. Each 
station handies its own keyboard scanning and screen refresh 
ana display. Communications with the 33-megabyte Winchester 
hard disk or the one-ha If -megabyte floppy disk are over the 
307-kilooaad RS 422 channel. Tne hard disk has an average 
access time of 50 milliseconds. Ti. ack- to- track access time 
is 1* milliseconds. The instantaneous transfer rate is al- 
most one-naif megabyte per second. The floppy disk has an 
average access time of 2dU milliseconds. TracK-to- track 
access time is 8 milliseconds, and the transfer rate is 
62.0 kilobytes per second. 
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Files are printed from a print spooler. The workstations 
can be performing other functions while a file is printinq. 
The printer can operate at up to 780 characters per second 
(CPS) in draft mode. H'vvrver, in a slower letter quality 
graphics mode and a letter-quality text mode, output suit- 
able for formal reports can be produced. System and appli- 
cation software consumes 8 meqaoytes of storaqe space on the 
33-megabyte Winchester disk, leavinq about 25 meqabytes for 
user files. 

The configuration can grow to a maximum of 16 stations sim- 
ply by addinq the cluster stations ana necessary communica- 
tion I/O cards. System mass storage can grow to 
100 meqabytes. For large development systems, a Convergent 
Technologies Meqaframe master processor can be used. This 
configuration will support up to 64 users and 10 giqabytes 
of online storage. NASTEC uses the Meqaframe as its devel- 
opment system but has not as yet placed any in the field. 

5.5.3 DATA BASE SIZING AND BACKUP 

The PACOR/GRO project intends to use the programmer/analyst 
workstation for every posible technical document. Arrange- 
ments have been made with the Technical Publications Depart- 
ment for assistance in formatting, online editinq, and 
reproduction from the output of the workstations. The size 
estimates given in Table 5-10 are based on a comparison of 
comparable documents from the Space Telescope Data Capture 
Facility and extrapolation from the portions of documents 
that are already on the workstations. 

Data base size estimates for the Spacelab project are shown 
in Table 5-11. They were derived by extrapolatinq the 
PACOR/GRO estimate to the total paqe count for Spacelab doc- 
umentation. 

Should online storaqe become scarce on a project, the fol- 
lowing strategies should be considered. 
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Table 5-10. PACOR/GRO Documentation Size Estimates 





5-45 






























Table 5-11. Spacelab Documentation Size Estimates 


Document 


Pages 


Requirements 

• Baseline Requirements 

• Hardware Specifications 

• Interface Control 

Design 

• Functional Design 

• Preliminary Design 

• Detailed Design 

Implementation 

• Operations Reference Manual 

• Programmer's Reference Manual 

• User's Guide 

• GINA Reference Manual 

Test 

• Test Plans 

• Test Procedures 

TOTAL 


3,000 

2,100 

5,500 

1,700 

12,300 


Total Disk Space Required 


12,300 x 22.36 



4b. 33 megabytes 
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First, on both the CASE 2000 and the Excelerator, online 
data base files can be backed up onto offline storage 
media. Since on most projects documents are developed se- 
quentially, many may be in progress simultaneously but would 
not all be changing at the same time. Strategies can thus 
be developed that allow for static documents and multiple 
copies of changing documents to be backed up separately and 
appropriately (in addition to periodic backups to protect 
against loss of online data due to disk failures) and re- 
stored online as and when needed, thereby providing online 
storage space relief. Backing up of files can be handled in 
two ways : 

• Individual files can be backed up on 0.5-megabyte 
(unformatted) floppy disks. For very large files 
or where an entire volume has to be saved at one 
time, this can be a time-consuming procedure. 

• Both the CASE 2000 and the Excelerator (IBM PC/XT) 
support a variety of standard communications 
protocols/emulators. The workstation ( s) can thus 
be linked to a mainframe computer and relatively 
inactive files down-loaded for backup purposes. 
Obviously, this option is subject to tne avail- 
ability of a mainframe computer. 

A second strategy to be considered is that online storage 
capacity can be expanded on the CASE 2000. Up to 120 mega- 
bytes of shared disk space can be configured on the master 
workstation. Furthermore, up to three additional 40- 
megabyte disks can be added to each cluster station. The 
latter would, however, be accessible only from the work- 
station to which they are attached. 

Although the IBM PC/XT can accommodate disk capacities 
higher than 10 megabytes, the Excelerator cannot access the 
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additional disk space. Excelerator workstations are, there- 
fore, currently limited to 10 megabytes of online storage. 

5.5.4 EVALUATION OF COSTS 

The cost figures presented here are list prices as of the 
summer of 1984. All of the CASE 2000 products must be pur- 
chased from NASTEC. For the Excelerator product, the basic 
computer nardware, an IBM PC/XT, can oe purchased Dy the 
user from any source desired. To that is added the Index 
Tecnnology Excelerator product. 

Excelerator 

Tne Excelerator model evaluated for this study ran on an IBM 
PC/XT configured as required with an IBM monochrome monitor 
and the high-resolution graphics card, a multifunction card, 
a mouse, ana Excelerator software from Index Technology. 
Because the user supplies the basic computer system and a 
printer, tne prices may vary depending on tne current supply 
channels. For a typical configuration with a moderate-speed 
printer, tne outside hardware would cost about $6000. To 
this must be added $9500 for the Excelerator package, for a 
total system price of $15,500. Tnis would provide the user 
with a system with a 360-kilobyte floppy disk drive, a 
10-megabyte (unformatted) Wincnester hard disk, and a 
160-CPS printer. Index Technology has a discount scnedule 
for volume purchases. For example, the price is reduced oy 
$500 for the second tnrough fifth purchases. If the 
Excelerator were to oe installed on an IBM PC/AT for im- 
proved performance, the price would be about $1100 more than 
for tne IBM PC/XT. This would provide a 1. 2-n.egaoyte floppy 
disk drive and a 20-megabyte Winchester hard disk. Addi- 
tional improvements in performance could be ootained by in- 
stalling a faster laser printer, at an additional cost of 
about $70UU per workstation. 
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The costs of the CASE 2000 vary dependinq on the configura- 
tion purchased. For a conf iguration of eight workstations, 
a one-half megabyte floppy disk, a 40-meqabyte (unformatted) 
Winchester hard disk, and a 780-CPS printer with system and 
DesignAid software, the total purchase price is $133,000. 
This provides a burdened price of $16,700 per workstation. 
Additional workstations can be added for the price of the 
station, additional memory, software license, and communica- 
tions card when necessary. NASTEC, like Index Technology, 
has a discount schedule for volume purchases. Depending on 
the volume purchased, cluster stations can be added for 
$ 11 , 100 . 

Comparison of Costs 

In qeneral, the cost per workstation for the Excelerator is 
the cost of one workstation multiplied by the number of 
workstations (minus volume discounts) . For the CASE 2000, 
the centralized data base architecture makes the cost-per- 
workstation computation very sensitive to the number of 
workstations (even excluding volume discounts) because the 
costs of the mass storage device(s) and printer(s) are pro- 
rated over the number of workstations. Thus, a direct cost 
comparison between the two is difficult except for specific 
configurations. 

For single-workstation configurations, the Excelerator is 
significantly cheaper than the CASE 2000. As the number of 
workstations increases, this cost advantage decreases, 
reaching a break-even point at about nine workstations. 
Beyond this number, the CASE 2000 offers a cost advantage. 
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SECTION 6 - CONCLUSIONS AND RECOMMENDATIONS 


The following conclusions have been drawn from the results 
of the programmer/analyst workstation evaluation presented 
in Section 5: 

• The workstations have a beneficial effect on pro- 
ductivity during the requirements analysis and design phases 
of a software development effort. This is found to be true 
even in the absence of graphics tools (i.e., with word proc- 
essors only) . 

• The workstations also have a positive effect on the 
quality of requirements analysis and design products. It is 
felt that this improvement in quality would further improve 
overall life-cycle productivity due to a potential reduction 
in testinq and maintenance efforts. 

• The Excelerator is easier to learn and use than the 
CASE 2000. 

• Feature for feature, the CASE 2000 is more powerful 
than the Excelerator. However, neither system is even close 
to being perfect. In particular, both systems provide 
limited validation and data dictionary support. 

It should be noted that the tools under consideration are 
still in their infancy. With advances in the state of the 
art of "intelligent" systems, both the Excelerator and the 
CASE 2000 (as well as others) are expected to become in- 
creasingly powerful tools to support requirements analysis 
and desiqn efforts. 

• The CASE 2000 architecture (specifically, the cen- 
tralized data base) lends itself to ease of configuration 
control and better analyst communication. The importance of 
this factor is felt to increase with project size and com- 
plexity . 
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• The Excelerator appears to be better suited to en- 
vironments in which one or more small problems are beinq 
worked on by a few people over relatively short periods of 
time . 

• The CASE 2000 appears to be better suited to en- 
vironments in which a single larqe and complex problem is 
beinq worked on by many people over relatively lonqer 
periods of time. 

Based on these conclusions, specific recommendations are 
presented in the following categories: 

• Overall recommendations 

• Recommendations for the PACOR/GRO project 

• Recommendations for other project usaqe 

• Long-term recommendations 

Overall Recommendations 

Overall recommendations are as follows: 

• The use of programmer/analyst workstations on soft- 
ware development projects is strongly recommended. 

• In the absence of such workstations, the use of 
word processors and other limited-capability tools 
(e.g., DFDraw , DBASE II, etc.) should be considered. 

• Neither the Excelerator nor the CASE 2000 is 
clearly superior to the other. Both, however, 
offer significant advantages over our current 
manual mode of analysis and desiqn. 

Recommendations for the PACOR/GRO Project 

The CASE 2000 is recommended for use on the PACOR/GRO proj- 
ect. This is a medium-sized, end-item deliverable, 32-month 
software development effort. It is envisioned that, durinq 
the requirements analysis and design phases, 10 to 
20 programmers and analysts will be involved and that 
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b workstations will be used. The recommendation is based on 
tne following: 

• The CASE 2000 centralized data base, which offers 
ease of configuration control and improved analyst 
communication 

• Comparable costs for an eight-workstation system 

• Current PACOR/GRO work available on the CASE 2000 

• Preference of PACOR/GRO personnel for the CASE 2000 

PACOR/GRO personnel's recommendations are included as Appen- 
dix E. These recommendations were presented to and accepted 
by CSC's internal source selection board. 

Recommendations for Other Projects 

This evaluation clearly establishes the benefits of using 
programmer/analyst workstations on software development 
projects. Which workstation is most suitable for a project 
depends, in large part, on the project goals and environment. 

The PC&A task's recommendations are included as Appendix E. 
The source selection board determined that tne Excelerator 
should be made available to the PC&A and Network Control 
Center projects. This recommendation was based on the fol- 
lowing : 

• The questionnaire showed the Excelerator to be pre- 
fer red 

• Ease of use of the Excelerator 

• Low costs for those projects with IBM PC/XTs 

• Preference of PC&A personnel for the Excelerator 
Long-Term Recommendations 

Because both the Excelerator and the CASE 2000 need improve- 
ments before eith'"' could be considered perfect and because 
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neither is clearly superior to the other, the followinq rec- 
ommendations are made: 

• Avoid lockinq ourselves into one or the other sys- 
tem at this time 

• Continue to evaluate both the Excelerator and the 
CASE 2000 by usinq them on different software de- 
velopment projects 

• Continue to evaluate new products as they become 
available 

• Continue to influence companies to extend their 
products in areas that are underdeveloped and that 
are needed 

• For those areas that companies are not willinq to 
pursue on their own, enter into aqreements that 
permit modifications and additions to standard 
products 
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APPENDIX A 


- PROGRAMMER/ANALYST WORKSTATION QUESTIONNAIRE 


PROCRAMMER/ ANALYST WORKSTATION 


(1 of 4) 


Evaluator Name: Data: 

Tool (Clrcla ona) : a) Excalarator b) Caaa 2000 


SECTION I ■ TOOL CAPABILITIES 

Plaaaa respond aa honestly aa poaalble to tha following questlone. In araaa that naad 
Improvement plaaaa provlda auggaatlona In tht comment* area an that vendors may evaluate 
enhancements. NA mean a not available or not applicable or not obaarved: 


Po you find thle tool: 


1. Provldaa Effective 
Graphics Support? 

2. Ia Eaay to Learn 
to Uaa? 

3. Provldaa Sufficiently 
Faat Reaponaa Time? 

4. Suppnrta DSDH/ProJect 
Requirements Analyala 
Methodology? 

5. Produce* Ouallty Data 
Flow Diagram? 

6. Supports PSDM/ProJect 
Dealgn Methodology? 

7. Produce* Ouallty 
Structure Chart*? 


Low Mien 

NA 1 2 3 4 5 

NA 1 2 3 4 5 

NA 1 2 3 4 5 

NA 1 2 3 4 5 

NA 1 2 3 4 5 

NA 1 2 3 4 5 

NA 1 2 3 4 5 


On what experience do you baae 
your a»ae*»me nt? 


Reading 


Observa- 
tion of 
Other* 


Tutorial 


Sllpht 

' Dae 


Exten- 

sive 

Us* 


8. Assists In the 
Generation and 
Maintenance of Data 
Dictionary Entrlea? 

9. User Friendly (Operator 
Interface)? 


NA 1 2 3 4 5 


NA 1 2 3 4 5 


10. Provides Additional NA l 2 3 4 5 

Project Management 
Tool*? 


11. Provide* QA Support? NA l 2 3 4 5 

12. Provides Consistency NA 1 2 3 4 5 

Checking ol a* itrements 

Analysis Productw? 

13. Provides Consistency NA 1 2 3 4 5 

Checklnc of Design 

Products’ 


i 
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SECTION 2 - GENERAL COMMENTS 


(2 of 4) 


1. Kim 5 upicti of chli cool you Ilka : 
(Highlit Importance) A. 


B. 


C. 


D. 


(Lowaat) E. 

2. Nan 5 aapacta of tnla cool you dislik e: 

(Highlit Dlallka) A. 

B. 

C. 

D. 

(Lowest Plaliki) E. 

3. I>o you find thia cool haa aaalacad you lo Improving tht quality of tha product? 

(Consider "quality" as attribute! (*.g., freedom from errors] that enhance the product'! 
value In later development stages Feel free to record your definition of quality 
In thin context.) 

Tea / No / Can't Tell 


4. Do you find the tool haa Improved Che apead at which you generate product!? 
(Conelder noth elapsed (calendar] time and work rate.) 

Elapsed Time: Faster/ Slower / Can't Tell 

Work Rate: Faster/ Slower / Can't Tell 


5. General Evaluation of Product: 


( 
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section 3 - rvALUAToK's histokt 


(3 of 4) 


1. Dagrae you conaldar yourself familiar with using tha product (clrcla on*); 

•) Not Vary b) familiar With Soma Aspects c) Familiar d) Knowltdgaabla 


product you have used: 

Indicate your degree of familiarity with 
the associated technique: 



Used 

Extensively 

Used 

Casual ly 

Studied, but 
iio actual use 

Not 

Familiar 

A) 

Data Flow Disarm* 

• 




b) 

Data Dictionary 





c) 

Structure Charts 





d; 

Word Processor 





*> 

Other Graphics 




• 

f> 

Language Compile:' 





R) 

?r Meet Management 
Tools 







3. Clrcl* tha rola you think la noat applicable to you: 

a) Division Evaluator 

b) Analyst 

c) Programmer 

d) QA 

a) S/'J Manat, *r 

f) Project Manager 

g) Support Personnel 


4. Years of Experience In Industry: 

With CSC: 


3. Length of tlx. you have used tool: 

(Estimate total hours or days 9 hours per day |av*raee|.) 
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SECTION 4 - MODES OF USE 


(4 of 4) 


1. In using Che workstation, did you work primarily from: 

a) relatively complete draft (of figures and text); 

, b) rough drafts (not complete enough to give to Tech. Pubs.); 

c) notes and Imagination (designing on-line). 

(If several modes, please describe your pattern of use) : 


2. Did you make more or fewer iterations of the crtiate/hardeopy/markup than you would 
without the workstation (how often did you get a clean copy?). 

a) more often b) about the same c) less often <*) can't tell 


3. Are you using the workstation primarily to: 

a) draw diagrams b) «■ tv re data c) both d) not sure? 
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APPENDIX B - CONTROL PROBLEM WORKSHOP PROBLEMS 
B.l TRAINING PROBLEM 

Develop a set of leveled data flow diagrams and a data dic- 
tionary for the following problem. If you have time, start 
to develop function specifications for low-level function 
bubbles in your data flows. 

A customer would like a simple message switch with the fol- 
lowing characteristics: 

The switch accepts messages. A message consists of at least 
a destination terminal identifier and a message text. The 
destination terminal identifier indicates to which of two 
destination terminals (T1 and T2) the message should be 
sent. The message text is a character string. 

Messages are sent from a single sender terminal, which is 
different from the two destination terminals. The sender 
terminal must be able to build a message in the format 
stated above. A destination terminal must be able to ex- 
tract the message text from a message. 

The switch acts as a buffer for the messages that are sent 
from the sender terminal to the destination terminals. Each 
destination terminal can receive only one messaqe at a time 
from the buffer. 

The switch should try to keep the destination terminals as 
busy as possible. Thus, the buffer should not be handled in 
a strict first-in-first-out (FIFO) manner. For example, if 
Tl is busy, and there are two messages in the buffer for T1 
and one messaqe for T2, the messaqe for T2 s'.ould be sent 
reqardless of the order in which the messaqe arrived. How- 
ever, the FIFO order should be preserved for each terminal. 

• 

The sender and terminals should be viewed as communicatinq 
with a source and sink external to the problem domain. The 
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messaqes sent and received should be archived so that a 
time-stamped history of events can be constructed at a fu- 
ture time. 

B. 2 CONTROL PROBLEM 

The Navy would like to buy a Heads-Up Display (HUD) subsys- 
tem for their A2D2 niqht/day/pursuit/support/attack/ 
reconnaissance/f iqhter/bomber aircraft. They have released 
an PFP that contains well over 100,000 paqes of detailed 
requirements coverinq all phases of the project, which we 
have summarized here. The pilot will view a display similar 
to that shown below: 


Target box 



Pilot's view of a typical heads*up display (HUD) 

The object of this display is to provide sufficient informa- 
tion content with low complexity. The correct scenario is 
for the pilot to fly the aircraft so that the selected tar- 
qet falls within the tarqet box; firinq durinq that time 
ensures a hit. Assuminq that the pilot has selected a 
trainable qun (that is, one that may be aimed automatically 
within a few deqrees) , the cursor inside the box points to 
jhere the qun is currently aimed. In addition, dependinq 
upcn the tarqet ranqe and type of armament selected, the 
tarqet box will vary in size, indicatinq the effective 
radius of the weapon; as the aircraft qets closer to the 
tarqet, the box qrows in size. The display also provides a 
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presentation of critical flight parameters, such as altitude 
and anqle of attack, plus a summary of the armament status. 
Furthermore, an arrow is superimposed on top of the selected 
tarqet; this not only assures the pilot that the correct 
tarqet is beinq tracked, but also presents the predicted 
tarqet direction of flight. 

Since the A2D2 has been around for a considerable lenqth of 
time, the HUD will have to fit into an existing framework of 
subsystems in the aircraft. Knowing how important the ex- 
ternal interfaces of a system are, we depart from our sum- 
mary format and reproduce the full text of the interface 
specifications contained in the Navy's 100,000+ pages. 

The other subsystems that the HUD must interface will in- 
clude the following: 

• ARMAMENT-SUBSYSTEM Controls weapon resources 

and targeting 

• NAVIGATION-SUBSYSTEM Includes all flight avionics 

equipment for aircraft 
quidance and control 

• TARGET-RADAR-SUBSYSTEM Acquires and tracks tarqet 

vehicles 

Brief, wasn't it? No mincinq of words in the Navy. To ex- 
pand on the Navy's interface specification sliqhtly, we 
should add that the Navy has other contractors ready, will- 
ing, and able to modify their subsystems (for a small fee) 
to provide you, within reason, with any information that you 
need to perform your job. You can further assume that it is 
definitely your job to specify whatever interface data you 
need. 

In reviewing the rest of the material in the RFP, we could 
not find much more information of a technical nature that 
seemed pertinent to the HUD, with the exception of a 
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paraqraph in the middle of paqe 78,483, which we reproduce 
here for your edification: 

"The pilot sees a display consistinq of fliqht param- 
eters, a target box with an aiming point, the actual 
target with state vector, and armament status. During 
the target engagement, these elements will change at 
arbitrary times, forcinq an immediate update of the dis- 
play. The pilot may terminate the HUD processing upon 
command . " 
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APPENDIX C - HARDWARE/SOFTWARE PROBLEM REPORT SUMMARY 

Hardware and software problems encountered during the evalu- 
ation period were loqqed and also reported to the vendors. 
Table C-l summarizes the total number of problems reported 
as well as the number fixed, either by a new software re- 
lease or a service call. Sections C.l and C.2 list the 
reported problems for Excelerator and the CASE 2000, respec- 
tively. 

Table C-l. Hardware/Software Problem Statistics 

Software Hardware 

Number of Number of Number of Number of 

Problems Problems Problems Problems 

Product Reported Fixed Reported Fixed 

Excelerator 7 1 6* 6* 

CASE 2000 6311 

*A11 six problems were related to the malfunctioning TECMAR 
board. 

C. 1 EXCELERATOR 

The following software and haraware problems were reported 
by users of the Excelerator. 

Software 

• Data flow diagrams print: While printinq a data 

flow diagram, a partial page is printed, then the 
system pauses and the remainder of the paqe is 
printed. When printing is resumed after the pause, 
two rows of pixels are repeated on the printout. 

• Restore function: If lonq and short project names 

are different and the short name is used, the re- 
store succeeds but the user (a) must enter 
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two carri .2 .a returns before the restore operation 
beqins anc. (b) cannot qet out of the restore func- 
tion without rebootinq the system. 

• Spooled print files not deleted: The spooled print 

file is not deleted after the file has been 
printed; the user must manually delete it usinq a 
Disk Operatinq System (DOS) command. 

• External entity description: When describinq an 

external entity and an ID of 11 characters is en- 
tered, a blank line is inserted in the external 
entity, then the ID. This causes the ID to run 
into the label in the box. 

• Word processor: When exitinq the Microsoft Word 

word processor, the system halts (fixed by a subse- 
quent release) . 

• Name chanqed: When selectinq the describe command 

and then pointinq to a data flow box, the name of 
the data flow on the qraph chanqed from 'captured- 
nondiqital-data ' to 'ca' . 

• Print problem: Only half of a data flow diaqram 

was printed. 

Hardware 

The followinq problems were fixed after the mult if unct ioninq 
TECMAR board was replaced. (The TECMAR multifunction board 
contains 256KB of random access memory, an RS-232 serial 
port, a parallel port, and a clock/calendar.) 

• Workstation hunq: After a cold start, the worksta- 

tion hunq while runninq the system self-test rou- 
tine, and a memory error messaqe was displayed. 

• Mouse inoperable: The Microsoft's mouse died in 

the middle of a qraphics session. 
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I, 

• Load problem: The Excelerator software could not 

load the DOS COMMAND processor. 

• Backup failed: The system went down during a proj- 

ect backup operation. 

• Access problem: Upon starting the Excelerator, 

neither the GRAPHICS nor the XLDICTIONARY submenus 
could be accessed. 

• Clock inoperable: The internal clock/calendar 

could not be reset. 

C. 2 CASE 2000 

The followinq software and hardware problems were reported 

by users of the CASE 2000.. 

Software 

• Horizontal lines drawn by usinq CODE+CURSOR are 
darker than vertical lines. 

• if a specification of the form ' @NAMEFILE ' is en- 
tered in response to the report prompt 'Object/ 
Relation NAME:', the DesiqnAid qenerates a report 
for the first object only and declares all other 
objects undefined. 

• User Validation: If the DesiqnAid is activated by 

CNTL+SHIFT+D , it fails to perform user validation 
and allows the invalid user to access the DesiqnAid. 

• Numerous data base access errors were encountered 
durinq interactive inquiry/update (fixed by a sub- 
sequent release) . 

• DesiqnAid error messaqes disappear from screen be- 
fore they can be read (fixed by a subsequent re- 
lease) . 
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• Concurrent DesiqnAid validation from two worksta- 
tions destroys file pointers (fixed by a subsequent 
release) . 

Hardware 

• Symbol key does not work (fixed by NASTEC via a 
service call) . 
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APPENDIX D - EVALUATOR SUGGESTIONS 


Over the evaluation period, users of the proqrammer/analyst 
workstations were encouraged to provide suggestions for sys- 
tem improvements. The suggestions received address three 
specific areas: software, hardware, and documentation. 

Sections D.l and D.2 list evaluator suggestions for the 
Excelerator and the CASE 2000, respectively. 

D.l EXCELERATOR 

The following suggestions were received from users of the 
Excelerator . 


Software 

• Allow validation of data flow diagrams across 
levels . 

• Allow for loose arrows in data flow diagrams. 

• Maintain user labels during a move object operation. 

• Allow for more flexibility in labeling and connect- 
ing objects. 

• Allow for user-defined ODjects. 

• Allow for word processing capabilities while manip- 
ulating text in the graphics mode. 

• Enhance the print screen utility so as to be able 
to predict what the finished graphic product will 
look like on the printed page. 

• Allow for moving more than one graphic object at a 
time . 

• Allow for a user-label option in a presentation 
graph. 
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• Enhance the system to (a) free the workstation 
while printing a graphics file and (b) delete a 
spooled graphics print file from the disk after the 
file has been printed. 

• Allow for more elaborate constructs in the data 
dictionary (e.q., OR, XOR, variable number of en- 
tries in a repeat group) . 

• Timestamp the data dictionary entries to enhance 
the merging process of data dictionary entries. 

• Allow for a more elaborate scheme than the data 
flow, record, item structure currently provide. 

• Allow for a shared data base among different work- 
stations. 

Hardware 

• Provide for a faster and better quality printer. 

i 

Documentation 

• Enhance the user guide and the tutorial. 

D. 2 CASE 2000 

) 

The following suggestions were received from users of the 

CASE 2000. 

Software 

• Enhance the man-machine interface to provide a 
higher degree of user friendliness: 

Enhance keyboard interface so as to reduce 
number of keystrokes required for an operation 

Provide mouse control 

Make system more menu driven 
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• Allow for automatic connections between objects 
instead of requiring users to specify the exact 
points (on the objects) to be connected. 

• Allow for loose arrows in data flow diaqrams. 

• Enhance the DesignAid to support multiple dic- 
tionaries. 

• Enhance the report-generating utility to qenerate 
better formatted reports. 

• Allow a valid qraphic symbol for the off-page con- 
nector. 

• Allow for more elaborate constructs in the data 
dictionary (e.q., OR, XOR, variable number of en- 
tries in repeat qroups) . 

• Enhance move and copy operations to perform the 
functions automatically instead of throuqh MARK/ 
BOUND. 

• Allow for moving of more than one object along with 
connectors . 

• Allow more space in the description field of the 
object definition menu. 

• Allow validation of data flow diagrams across 
levels . 

• Enhance DesiqnAid to recognize processes that are 
black holes or divine entities. 

Hardware 

• None 
Documentation 

• Enhance the reference manuals and tutorials. They 
are often incorrect and do not describe all the 
available capabilities. 
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APPENDIX E ~ PACOR/GRO PROJECT RECOMMENDATIONS TO 
THE SOURCE SELECTION BOARD 


1. Either CASE 2000 or Excelerator is preferable to 

manual approach 

2. Both have a lonq way to qo 

3. CASE 2000 preferable for PACOR/GRO project 

• CASE 2000 centralized data base is a biq ad- 
'vantaqe; qives more flexible and sophisticated 
reports; makes CM possible 

• Excelerator merqe is unsophisticated, compli- 
cated, and errorprone; PACOR/GRO project is 
already at diskette limit for data dictionary 

• CASE 2000 quasi-proqrammable; at first is more 
difficult to learn; later more powerful; help 
is readily available 

• CASE 2000 more flexible; features can be 
tailored to qenerate the product you want; 
qraphics can be moved and sized to specific 
need 

• CASE 2000 word processor avialable in all 
products; Excelerator cannot mix text and 
qraphics 
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COMPUTER SCIENCES CORPORATION 


INTEROFFICE CORRESPONDENCE 

to: Source Selection from: G. Page uf\' date: October 18, 1984 

Committe mail code/ext: / ” ref: 

subject: Workstation Recommendation 


Recommendation: Select the Excelerator Programmer/Analyst 

Workstation 

The following factors are the basis for the recommendation. 

1. The Excelerator is easier to learn and easier to use, i.e., 
it is fun to use. 

Benefits : 

• It will be accepted more quickly 

• It will minimize phase-in, learning, training 

• Therefore, it will improve productivity 

2. The Excelerator, which uses an IBM PC, provides greater 
hardware flexibility and usability in other contexts. 

Benefits : 

• The IBM PCs could be used for other functions in slow 
periods . 

• The Index Technologies' product could be replaced by 
a future, better IBM PC-compatible product 

• The Index Technologies' product could be replaced by 
upgrading or trading for an IBM PC/G or PC/GX when 
the appropriate software is available, assuming it 
is a better product. 

3. The Excelerator uses a newer technology. 

Benefit : 

• Evolutionary enhancement and growth of capability to 
satisfy our needs is more likely 

4. Even though the Case 2000 has been in the field for a longer 
period of time, conceptually, the Case 2000 does not ade- 
quately address the problem we want to solve, i.e., it does 
not help the programmer/analyst enough. 

Penalties: 

• Satisfactory evolutionary enhancement and growth of 
capability to satisfy our needs is not likely. 

• It will not become significantly easier to learn or 
use . 
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5. Most users judge the overall capability of Excelerator 
to be better. However, they feel that either one would 
offer improvements in productivity, quality, and reli- 
ability. 

Benefit: 

• There will be improvements in productivity, quality, 
and reliability because a useful tool has been pro- 
vided. However, the improvements from using the 
Case 2000 will not be as great (because of 1 above) 
nor will they escalate significantly (because of 4 
above) . In addition, there may be a detrimental 
antagonistic attitude from those who participated 

in the evaluation if Case 2000 is chosen (because 
of 1 above) . 

Penalty: 

• By deciding to use Case 2000, there will be little 
opportunity to cost-effectively upgrade to a newer, 
better product (because of 2 above) . 

Recommendation: Select the Excelerator Programmer/Analyst 

Workstation 

GP :gsp 

Copy: M . Plett 






